Large pre-trained models, such as Bert, GPT, and Wav2Vec, have demonstrated great potential for learning representations that are transferable to a wide variety of downstream tasks . It is difficult to obtain a large quantity of supervised data due to the limited availability of resources and time. In light of this, a significant amount of research has been conducted in the area of adopting large pre-trained datasets for diverse downstream tasks via fine tuning, linear probing, or prompt tuning in low resource settings. Normalization techniques are essential for accelerating training and improving the generalization of deep neural networks and have been successfully used in a wide variety of applications. A lot of normalization techniques have been proposed but the success of normalization in low resource downstream NLP and speech tasks is limited. One of the reasons is the inability to capture expressiveness by rescaling parameters of normalization. We propose KullbackLeibler(KL) Regularized normalization (KL-Norm) which make the normalized data well behaved and helps in better generalization as it reduces over-fitting, generalises well on out of domain distributions and removes irrelevant biases and features with negligible increase in model parameters and memory overheads. Detailed experimental evaluation on multiple low resource NLP and speech tasks, demonstrates the superior performance of KL-Norm as compared to other popular normalization and regularization techniques.

在机器人和人类运营商之间分享自主权可以促进机器人任务示范的数据收集，以不断改进学习模型。然而，沟通意图的手段和关于未来的原因是人类和机器人之间的差异。我们介绍了辅助Tele-Op，虚拟现实（VR）系统，用于收集展示自主轨迹预测的机器人任务演示，以传达机器人的意图。随着机器人移动，用户可以在需要时切换自主和手动控制。这允许用户通过高成功率和比手动遥操作系统更轻松地收集任务演示。我们的系统由变压器供电，可以为未来提供潜在的状态和行动的窗口 - 几乎没有添加计算时间。密钥识别是，如果用户决定模型预测的操作是不合适的，则可以在变换器序列内的任何位置注入人类意图。在每次步骤中，用户可以（1）无所作为并允许自主操作在观察机器人的未来计划序列时继续，或者（2）接管并暂时规定不同一组动作以使模型返回到轨道上。我们在https://sites.google.com/view/assistive-teleop上托管视频和其他补充材料。

Current image generation models struggle to reliably produce well-formed visual text. In this paper, we investigate a key contributing factor: popular text-to-image models lack character-level input features, making it much harder to predict a word's visual makeup as a series of glyphs. To quantify the extent of this effect, we conduct a series of controlled experiments comparing character-aware vs. character-blind text encoders. In the text-only domain, we find that character-aware models provide large gains on a novel spelling task (WikiSpell). Transferring these learnings onto the visual domain, we train a suite of image generation models, and show that character-aware variants outperform their character-blind counterparts across a range of novel text rendering tasks (our DrawText benchmark). Our models set a much higher state-of-the-art on visual spelling, with 30+ point accuracy gains over competitors on rare words, despite training on far fewer examples.

We present Mu$^{2}$SLAM, a multilingual sequence-to-sequence model pre-trained jointly on unlabeled speech, unlabeled text and supervised data spanning Automatic Speech Recognition (ASR), Automatic Speech Translation (AST) and Machine Translation (MT), in over 100 languages. By leveraging a quantized representation of speech as a target, Mu$^{2}$SLAM trains the speech-text models with a sequence-to-sequence masked denoising objective similar to T5 on the decoder and a masked language modeling (MLM) objective on the encoder, for both unlabeled speech and text, while utilizing the supervised tasks to improve cross-lingual and cross-modal representation alignment within the model. On CoVoST AST, Mu$^{2}$SLAM establishes a new state-of-the-art for models trained on public datasets, improving on xx-en translation over the previous best by 1.9 BLEU points and on en-xx translation by 1.1 BLEU points. On Voxpopuli ASR, our model matches the performance of an mSLAM model fine-tuned with an RNN-T decoder, despite using a relatively weaker sequence-to-sequence architecture. On text understanding tasks, our model improves by more than 6\% over mSLAM on XNLI, getting closer to the performance of mT5 models of comparable capacity on XNLI and TydiQA, paving the way towards a single model for all speech and text understanding tasks.

End-to-end text-to-speech (TTS) systems have been developed for European languages like English and Spanish with state-of-the-art speech quality, prosody, and naturalness. However, development of end-to-end TTS for Indian languages is lagging behind in terms of quality. The challenges involved in such a task are: 1) scarcity of quality training data; 2) low efficiency during training and inference; 3) slow convergence in the case of large vocabulary size. In our work reported in this paper, we have investigated the use of fine-tuning the English-pretrained Tacotron2 model with limited Sanskrit data to synthesize natural sounding speech in Sanskrit in low resource settings. Our experiments show encouraging results, achieving an overall MOS of 3.38 from 37 evaluators with good Sanskrit spoken knowledge. This is really a very good result, considering the fact that the speech data we have used is of duration 2.5 hours only.

Our education system comprises a series of curricula. For example, when we learn mathematics at school, we learn in order from addition, to multiplication, and later to integration. Delineating a curriculum for teaching either a human or a machine shares the underlying goal of maximizing the positive knowledge transfer from early to later tasks and minimizing forgetting of the early tasks. Here, we exhaustively surveyed the effect of curricula on existing continual learning algorithms in the class-incremental setting, where algorithms must learn classes one at a time from a continuous stream of data. We observed that across a breadth of possible class orders (curricula), curricula influence the retention of information and that this effect is not just a product of stochasticity. Further, as a primary effort toward automated curriculum design, we proposed a method capable of designing and ranking effective curricula based on inter-class feature similarities. We compared the predicted curricula against empirically determined effectual curricula and observed significant overlaps between the two. To support the study of a curriculum designer, we conducted a series of human psychophysics experiments and contributed a new Continual Learning benchmark in object recognition. We assessed the degree of agreement in effective curricula between humans and machines. Surprisingly, our curriculum designer successfully predicts an optimal set of curricula that is effective for human learning. There are many considerations in curriculum design, such as timely student feedback and learning with multiple modalities. Our study is the first attempt to set a standard framework for the community to tackle the problem of teaching humans and machines to learn to learn continuously.

Finetuning language models on a collection of datasets phrased as instructions has been shown to improve model performance and generalization to unseen tasks. In this paper we explore instruction finetuning with a particular focus on (1) scaling the number of tasks, (2) scaling the model size, and (3) finetuning on chain-of-thought data. We find that instruction finetuning with the above aspects dramatically improves performance on a variety of model classes (PaLM, T5, U-PaLM), prompting setups (zero-shot, few-shot, CoT), and evaluation benchmarks (MMLU, BBH, TyDiQA, MGSM, open-ended generation). For instance, Flan-PaLM 540B instruction-finetuned on 1.8K tasks outperforms PALM 540B by a large margin (+9.4% on average). Flan-PaLM 540B achieves state-of-the-art performance on several benchmarks, such as 75.2% on five-shot MMLU. We also publicly release Flan-T5 checkpoints, which achieve strong few-shot performance even compared to much larger models, such as PaLM 62B. Overall, instruction finetuning is a general method for improving the performance and usability of pretrained language models.

在本文中，我们通过神经生成编码的神经认知计算框架（NGC）提出了一种无反向传播的方法，以机器人控制（NGC），设计了一种完全由强大的预测性编码/处理电路构建的代理，体现计划的原则。具体而言，我们制作了一种自适应剂系统，我们称之为主动预测性编码（ACTPC），该系统可以平衡内部生成的认知信号（旨在鼓励智能探索）与内部生成的仪器信号（旨在鼓励寻求目标行为）最终学习如何使用现实的机器人模拟器（即超现实的机器人套件）来控制各种模拟机器人系统以及复杂的机器人臂，以解决块提升任务并可能选择问题。值得注意的是，我们的实验结果表明，我们提出的ACTPC代理在面对稀疏（外部）奖励信号方面表现良好，并且具有竞争力或竞争性或胜过几种强大的基于反向Prop的RL方法。

机器学习中的超参数优化通常是使用只会导致大约一组超参数的幼稚技术来实现的。尽管贝叶斯优化之类的技术在给定超参数的给定域进行了智能搜索，但不能保证最佳解决方案。大多数这些方法的一个主要缺点是用超参数数量增加其搜索域的指数增加，从而增加了计算成本并使方法缓慢。超参数优化问题本质上是双重优化任务，一些研究尝试了解决此问题的双重解决方案方法。但是，这些研究假设了一组独特的模型权重，可以最大程度地减少训练损失，这通常受到深度学习体系结构的影响。本文讨论了一种基于梯度的双层方法，该方法解决了这些缺点以解决超参数优化问题。所提出的方法可以处理我们在实验中选择正则化高参数的连续超参数。该方法保证了本研究已在理论上证明的一组最佳超参数的收敛。该想法基于使用高斯过程回归近似较低级别的最佳值函数。结果，使用增强拉格朗日方法解决的单个级别约束优化任务缩小为单个级别约束优化任务。我们已经对多层感知器和LENET架构进行了有关MNIST和CIFAR-10数据集的广泛计算研究，以证实该方法的效率。一项针对网格搜索，随机搜索，贝叶斯优化和Hyberband方法的比较研究表明，所提出的算法会收敛于较低的计算，并导致模型在测试集上更好地推广。

基于会话的建议系统在会话中捕获用户的短期兴趣。会话上下文（即，会话中用户在会话中的高级兴趣或意图）在大多数数据集中都没有明确给出，并且隐式推断会话上下文作为项目级属性的汇总是粗略的。在本文中，我们提出了ISCON，该ISCON隐含地将会议上下文化。ISCON首先通过创建会话信息图，学习图嵌入和聚类来为会话生成隐式上下文，以将会话分配给上下文。然后，ISCON训练会话上下文预测器，并使用预测上下文的嵌入来增强下一项目的预测准确性。四个数据集的实验表明，ISCON比最新模型具有优越的下一项目预测准确性。REDDIT数据集中的ISCON的案例研究证实，分配的会话上下文是独特而有意义的。